LINDA DARLING-HAMMOND Reforming Teacher Preparation and Licensing: Debating the Evidence

This article responds to Dale Ballou and Michael Podgursky's claims that the National Commission on Teaching and America's Future has misrepresented research data and findings. After reviewing and responding to each of their charges, the article indicates the ways in which their critique itself has misreported data and misrepresented the Commission's statements and recommendations. Ballou and Podgursky ignore and misconstrue the research evidence presented by the Commission in support of its key conclusions. Following an analysis of the ways in which the critique misrepresents the findings from research on teacher education to bolster the argument that training for teaching is unnecessary, this reply offers an argument for professional teaching standards as a key factor in achieving greater equity and excellence in American schools. What Matters Most, the 1996 report of the National Commission on Teaching and America's Future (the Commission), was the product of a 26-member bipartisan panel of governors, legislators, business leaders, community leaders, and educators. The Commission was concerned with understanding what it would take to enable every child in America to reach the new high standards of learning being enacted by states across the nation. Following two years of intensive study and debate, the Commission concluded that recent reforms like new curriculum standards, tests, and accountability schemes are unlikely to succeed without major investments in teaching. The history of school reform has illustrated that innovations pursued without adequate investments in teacher training have failed time and again. Furthermore, the demands of new subject matter standards and much more diverse student bodies require deeper content knowledge, more sophisticated pedagogical and diagnostic skills, and broader repertoires of teaching strategies for teachers. The Commission report offered an interlocking array of 22 recommendations aimed at ensuring "a caring, competent, and qualified teacher" for every child, working in schools organized to support their success. The Commission's recommendations have stimulated legislative initiatives to improve teaching in more than 25 states, in many more local districts, and at the federal level. The recommendations also provide the basis for ongoing work with 15 state partners, more than 40 national partner organizations, and 9 urban districts that are working together to develop more effective systems for recruiting, preparing, inducting, evaluating, and supporting teachers and for redesigning schools so that they can support more powerful learning for a wider range of students. As the Commission's implementation efforts have unfolded, attacks on its work have been circulated by Dale Ballou and Michael Podgursky through the Government Union Review (1997a) and Chester Finn's Thomas B. Fordham Foundation (1999). The major substantive points of disagreement revolve around two issues: Ballou and Podgursky, along with Finn and other contributors to the Fordham Foundation (1999) "manifesto," object to standards for teaching or teacher education that would influence the knowledge and skills teachers are expected to acquire or that schools of education might be expected to impart. Instead they prefer that administrators select teachers from the open market and evaluate them according to students' test scores. In addition, they object to the idea that school resources might contribute as much to student achievement as student background characteristics such as race or parent income, and that states therefore have an obligation to equalize access to those resources. They are content to allow the market to operate unrestrained in the face of inequities in access to dollars, qualified teachers, curriculum, and class sizes that contribute to disparate outcomes between advantaged and disadvantaged students. Their proposed solutions would continue and exacerbate these disparities by creating additional disincentives for the most able teachers to work with the most needy students, by removing requirements for even minimally equitable treatment of less advantaged students in the allocation of teachers and other resources, and by reducing teachers' access to knowledge that might help them to become more effective. In past critiques, Ballou and Podgursky have argued that the Commission report was the work of education insiders, ungrounded in research, and likely to shift control from governments to private organizations. In this volume of the Teachers College Record, Ballou and Podgursky (1999a) go further to charge, falsely in each instance, that the Commission has misrepresented research data and findings. In the course of their argument, their critique itself misreports data, misrepresents the Commission's statements and recommendations, and variously ignores and misconstrues the research evidence presented in support of the report's key findings. In addition, the authors misrepresent the findings of research on teacher education in an effort to argue that training for teaching is unnecessary. This reply offers evidence on the key points of contention. BALLOU AND PODGURSKY'S MISREPRESENTATIONS The commission's membership and recommendations. In the first pages of their article, Ballou and Podgursky mischaracterize the membership of the Commission, describing it as "representatives of various education interest groups, including the presidents of the National Education Association, the American Federation of Teachers, the National Council for the Accreditation of Teacher Education (NCATE), and the National Board for Professional Teaching Standards (NBPTS)." This statement completely ignores the other 22 members of the Commission, including its Democratic and Republican governors, two state legislators, a current and a former state superintendent, three business leaders, two community leaders, two college presidents, two leaders of school reform initiatives, an education dean and a professor, a local superintendent, principal, and three classroom teachers. Despite the breadth of the group as a whole, because these four education organizations were represented, Ballou and Podgursky (1997) have suggested elsewhere that the Commission was uncritically supportive of unions and of professional certification and accreditation. Ballou and Podgursky also ignore 19 of the Commission's 22 recommendations, including those that address student standards; new models of teacher preparation, induction, and professional development; recruitment reforms, including streamlined hiring procedures, reciprocity in licensing, and alternative pathways for mid-career entrants; rewards for teacher knowledge and skill, including recognition of expert teachers and strategies to remove incompetent teachers; and redesigned schools that allocate more resources to the front lines of teaching. The recommendations emerged from a 2-year process of research and vociferous debate in which each had to pass muster with the entire group of diverse, strong-willed commissioners, which was no easy task. Ballou and Podgursky's (1997a) allegations that the Commission offered a narrow agenda controlled by the education establishment ignores the fact that the report included recommendations that have previously been opposed by the same groups they suggest controlled the agenda, including changes in union policies that protect incompetent teachers and university practices that undermine teacher quality. Ballou and Podgursky's one-sided treatment of the Commission's proposals reflects the ideological lens they apply to the work. The definition of teacher expertise. Ballou and Podgursky accuse the Commission of misrepresenting research findings because we use the term teacher "expertise" to discuss a group of variables together (e.g., teacher test scores, master's degrees, and experience). They seem to be suggesting that the only variable that should properly be reported as a measure of expertise is the percentage of teachers with master's degrees, and they discuss the issue as though the Commission's major recommendation regarding the preparation of teachers is to require master's degrees. They then look at large-scale quantitative studies for support for the idea that master's degrees are a major determinant of student achievement. Their argument makes little sense. First of all, the notion that master's degrees are the best or only measure of expertise is one that exists only in Ballou and Podgursky's imagination. It is not a notion put forth by the Commission, which recommends a comprehensive array of measures to improve teachers' qualifications and access to knowledge, including policies and practices influencing admissions to teacher education; college studies in the liberal arts, subject matter, and education; extended student teaching or internships in professional development schools; licensing tests of both subject matter knowledge and teaching skills; induction for beginning teachers; and ongoing professional development. While the Commission spoke approvingly of new 5-year teacher education models that have produced better entry and retention rates and evidence of effectiveness (see below) and recommended that teacher education extend beyond the typical 4-year undergraduate program to provide time for a year-long internship, it did not suggest that states require master's degrees for teaching, either at the preservice or inservice levels. In fact, several of the programs the Commission described as high quality (including 5thyear models) do not require a master's degree. The goal is to construct an interlocking set of policies that encourage high-ability individuals into teaching, provide them access to knowledge that will improve their effectiveness, and keep them in the classroom so that their knowledge and experience become a resource for children's learning. Second, research on teaching has developed a view of teaching expertise that includes general knowledge and ability, verbal ability, and subject matter knowledge as foundations; abilities to plan, organize, and implement complex tasks as additional factors; knowledge of teaching, learning, and children as critical for translating ideas into useful learning experiences; and experience as a basis for aggregating and applying knowledge in nonroutine situations. David Berliner's studies of expertise in teaching, for example, include experience along with several of these other traits as a critical aspect of expertise (see, e.g., Berliner, 1986). All of these factors combine to make teachers effective; furthermore, one cannot fully partial out the effects of one factor as opposed to another as many are highly correlated. It is because expertise is a product of many kinds of knowledge and skill that the Commission recommended a balanced, multifaceted preparation for teaching. Third, given the wide variability in the content of master's degrees pursued by teachers, it would be difficult to describe them as representing any common body of knowledge or "treatment." Few teachers have pursued master's degrees through a coherent program of preservice education tightly linking content, content pedagogy, and clinical training, like the 5-year programs the Commission described. Some master's degrees have been directly related to teaching and represent a coherent program of studies, for example, those in reading or special education most commonly taken by elementary teachers. (Interestingly, many studies find a larger effect of master's degrees on student achievement at the elementary level and in reading, although one should not make too much of this fact, given the points discussed below.) Other master's degrees have been a collection of courses that may or may not address teachers' abilities in the classroom to a great extent. Many have been pointed at jobs outside of teaching, such as administration, guidance counseling, measurement and evaluation, and the like. Thus, there is reason to expect that some master's degree studies would affect teaching ability, but not much reason to expect the effect of master's degrees as an undifferentiated variable to be uniform or large in the aggregate. The Commission suggested that master's degrees for which salary credit is awarded be more directly focused on improving teachers' classroom teaching knowledge and skills, in part because they have not always been in the past. Finally, it would be particularly odd to find a large effect of master's degrees in a multivariate study of the sort Ballou and Podgursky focus on here. For one thing, variables describing teacher expertise are highly correlated; teachers with higher test scores and those with greater experience are much more likely to have a master's degree. Other variables frequently examined in large scale studies, such as certification status, are also correlated with master's degrees. As Ballou and Podgursky understand, variable collinearity means that when two or more highly correlated variables are entered into an equation, one absorbs some of the variance that might otherwise appear to be explained by another. Thus, one cannot disaggregate with precision the effects of one such variable from those of another that is closely related to it. Their suggestion that the Commission should have done this in order to "prove" the effects of master's degrees--a single crude proxy for a wide array of knowledge and skills--contradicts their admonitions elsewhere in the paper. Ferguson's Texas study. Ballou and Podgursky object to the Commission's description of Ronald Ferguson's (1991) study of 900 Texas school districts, which found a large influence of teacher qualifications on student achievement at the district level. First, as described above, they object to the Commission's labeling as "teacher expertise" the set of variables that include teachers' scores on the Texas licensing test (TECAT), teacher education (master's degrees), and teacher experience. We believe that all of these variables and others may capture aspects of teacher expertise that interact with one another, and we do not hold special warrant for master's degrees as the central indicator of expertise. In studies using large quantitative data sets in which all of the variables are crude proxies for more nuanced conditions, including many that are unmeasured, one can only speculate about the finer grained phenomena that are at work in the real world. Our point in presenting the Ferguson findings was not to argue for master's degrees in education, or for any specific teacher education approach (we address these ideas elsewhere in the Commission's reports using different evidence bases, described later in this article), but to argue that policies that aim to enhance teachers' abilities and the equitable distribution of well-qualified teachers are important. Ballou and Podgursky note correctly that the TECAT scores, given the other variables in the equations, represent the lion's share of the variance attributable to teachers' characteristics in Ferguson's estimates. Having described the TECAT as a test of basic literacy, they want to argue that the only implication of Ferguson's findings is that it may be important to have literate teachers, and "is there anyone who thinks otherwise?" Clearly policy makers who constructed a system of teacher hiring in Texas that did not distribute teachers equitably on this or other measures must have thought otherwise. That is the point of the Commission's recommendations and the import of Ferguson's findings: that teacher qualifications matter in policy decisions about resource allocations and school improvement. A critical policy question is what Ferguson's finding, necessarily based on an easily available, large-scale quantitative measure, actually represents. The TECAT is actually a somewhat broader test than Ballou and Podgursky indicate, as it tests verbal ability, logical thinking and research skills, and professional knowledge. Knowing the content of the test helps only a bit in drawing inferences from these findings, however. A study of this kind can indicate only that this measure of teachers' general and professional knowledge and skills--and whatever unmeasured attributes it may correlate with--is strongly associated with average student achievement. As Ferguson notes: We can only speculate about what teachers with high scores do differently from teachers with low scores. One possibility is that teachers with higher scores more often attended colleges that were effective at preparing them to become good teachers. Alternatively, or in addition, teachers with higher scores may have thinking habits that make them more careful in preparation of lesson plans or more articulate in oral presentation. (p. 477) Other kinds of research are needed (some are cited in the Commission's report; others are suggested elsewhere; see, e.g., Darling-Hammond, 1998) to figure out what programs and policies are likely to make a meaningful difference in the kinds of knowledge and skills that have been found in finer grained studies to be important for effective teaching. In addition to their own narrow reading of the term teacher expertise, Ballou and Podgursky want to take issue with Ferguson's finding that school inputs, including teachers, may account for as much or more of the variance in student achievement among districts as do student background variables. They acknowledge that such a finding is "extraordinary," and note, "It appears to controvert a long-standing belief among researchers, dating back to the work of James Coleman in the 1960s, that students' family backgrounds have far more explanatory power as predictors of achievement than schooling variables." Ballou and Podgursky look for ways to buttress this long-standing belief. Calling the Commission's account of the research "incorrect and misleading," they first complain about the level of aggregation of the data. Their observation that "the same teacher qualifications may account for a very different share of explained variation in individual student achievement" than they do when assessing interdistrict variation is obvious and uncontested. The Commission accurately noted in its summary of the findings that this was a district level study. The research studied what it studied with the data that were available and produced interesting and important findings. The fact that Ballou and Podgursky do not like the findings because they challenge presumptions about school inputs and student backgrounds does not make them invalid or their presentation misleading. Ironically, later in their paper Ballou and Podgursky try to establish even more sweeping claims about the relationships between policies and student achievement using student achievement data aggregated to the state level with a bizarre study that includes independent variables that do not measure teacher qualifications and that postdate their achievement data. (We discuss this strange analysis later.) Ballou and Podgursky take issue with the Commission's statement that in the study's various estimates of influences on average student achievement, teachers' expertise "accounted for about 40% of the measured variance ... more than any other single factor." The 40% statistic was derived by summing the percentages offered in the article for variance explained by teachers' scores, teacher experience, and master's degrees, and by examining published and unpublished equations showing the combined variables accounting for between 38% and 43% of the variance, after taking account of the set of other controls in the estimates. While Ballou and Podgursky are technically correct that issues of covariance make it impossible to precisely assign proportions of variance to variables as though they are wholly orthogonal, our procedure was not unreasonable and the estimate not far afield as a simple summary to enable readers to gauge the relative influence of these variables compared to others measured. The point we correctly made is that this study demonstrates that teachers are very important--at least comparable in importance to parents or other family background factors when making comparisons across districts. While this finding may be "extraordinary" in light of the presumption that school inputs don't matter, it is not unique. Another study using licensing test scores (Strauss & Sawyer, 1986) found similarly strong influences of teacher quality (as measured by teachers' scores on the National Teacher Examinations that measure subject matter and pedagogical knowledge) on student test performance at the district level. A more recent Texas study (Fuller, 1999) found that students of fully licensed teachers were significantly more likely to pass the Texas state achievement tests, after controlling for student socioeconomic status, school wealth, and teacher experience. In a recent school level analysis of mathematics test performance in California high schools, Fetler (1999) found a strong negative relationship between student scores and the percentage of teachers on emergency certificates, as well as a smaller positive relationship between student scores and teacher experience levels, after controlling for student poverty rates. Ferguson and Ladd's Alabama study. The Commission cited Ferguson and Ladd's (1996) study of student achievement in Alabama as additional support for the proposition that school inputs, particularly teachers (and to a lesser extent class sizes), matter for student learning. Ballou and Podgursky do not take issue with the findings as cited by the Commission. Their complaint about this study again concerns the Commission's definition of expertise, which included ACT scores, teacher experience, and master's degrees together. Here, too, the problem lies with Ballou and Podgursky's own narrow definition of the term, their misrepresentation of the Commission's proposals, and their desire to find malfeasance where none exists. Greenwald, Hedges, and Laine's review. Ballou and Podgursky charge that the Commission's description of a meta-analysis of education production function studies by Greenwald, Hedges, and Laine (GHL) (1996) is "inaccurate." They once again misrepresent the Commission's actual statements when they claim: "The NCTAF describes this as a review of 'sixty production function studies,' and summarizes the GHL findings in a chart that purports to show the gain in student test scores per $500 invested in various educational reforms.... The relevant table in GHL shows that the findings on teacher education are based not on 60 studies but eight." In fact, the meta-analysis did include 60 primary studies and the Commission's summary of its major findings (below) is drawn directly from the study's conclusions and is entirely accurate. The Commission stated: These findings about the influences and relative contributions of teacher training and experience levels are reinforced by those of a recent review of 60 production function studies (Greenwald, Hedges, and Laine, 1996) which found that teacher education, ability, and experience, along with small schools and lower teacherpupil ratios, are associated with increases in student achievement across schools and districts. In this study's estimate of the achievement gains associated with expenditure increments, spending on teacher education swamped other variables as the most productive investment for schools (see figure 5). (Darling-Hammond,